# Diffusion Transformer

InstantCharacter is a character personalization framework based on diffusion transformers, designed to overcome the limitations of existing learning-based customization methods. The framework's main advantages lie in its open-domain personalization, high-fidelity results, and effective character feature processing capabilities, suitable for generating various character appearances, poses, and styles. The framework utilizes a large-scale dataset containing tens of millions of samples for training to achieve both character consistency and text editability optimization. This technology sets a new benchmark for character-driven image generation.

EasyControl is a framework that provides efficient and flexible control for Diffusion Transformer (DiT), aiming to solve the efficiency bottlenecks and lack of model adaptability in the current DiT ecosystem. Its main advantages include: supporting multiple conditional combinations, improving generation flexibility and inference efficiency. This product is developed based on the latest research results and is suitable for use in image generation, style transfer, and other fields.

DiTCtrl is a video generation model based on the Multimodal Diffusion Transformer (MM-DiT) architecture, focusing on generating coherent scene videos with multiple continuous prompts without additional training. By analyzing the attention mechanism of MM-DiT, this model achieves precise semantic control and attention sharing between different prompts, producing videos with smooth transitions and cohesive object movement. The main advantages of DiTCtrl include no training requirement, capability to handle multi-prompt video generation tasks, and showcasing cinematic transition effects. Additionally, DiTCtrl introduces a new benchmark called MPVBench specifically designed for evaluating the performance of multi-prompt video generation.

OminiControl is a minimal yet powerful universal control framework designed for diffusion transformer models like FLUX. It supports subject-driven control and spatial tasks (such as edge guidance and image restoration). The design of OminiControl is extremely streamlined, introducing only 0.1% additional parameters to the base model while retaining the original model structure. This project is developed by the Learning and Vision Laboratory at the National University of Singapore, representing the latest advancements in image generation and control technologies in the field of artificial intelligence.

MelodyFlow is a high-fidelity music generation and editing model based on text control. It utilizes continuous latent representation sequences to avoid information loss associated with discrete representations. Built on a diffusion transformer architecture and trained with flow matching objectives, the model can generate and edit a diverse range of high-quality stereo samples while maintaining the simplicity of text descriptions. MelodyFlow also explores a novel regularized latent inversion method for text-guided editing in zero-shot testing, demonstrating its superior performance across various music editing prompts. The model has been evaluated using objective and subjective metrics, confirming that it matches the quality and efficiency of established benchmarks in standard text-to-music evaluations while surpassing previous state-of-the-art techniques in music editing.

3DTopia-XL is a high-quality 3D asset generation technology built on the Diffusion Transformer (DiT), utilizing a novel 3D representation method called PrimX. This technology encodes 3D shapes, textures, and materials into a compact N x D tensor, where each element represents a volumetric primitive anchored to the surface of the shape, encoding signed distance fields (SDF), RGB, and materials through voxelized payloads. This process generates 3D PBR assets from text/image inputs in just 5 seconds, suitable for graphics pipelines.

Qihoo-T2X is an open-source project developed by 360CVGroup, representing an innovative paradigm of diffusion transformer (DiT) architecture for text-to-any-task (Text-to-Any). The project aims to enhance processing efficiency through proxy token technology. Qihoo-T2X is an ongoing project, with a team committed to continuously optimizing and enhancing its functionalities.

Tora is a video generation model based on Diffusion Transformers (DiT), which integrates text, visual, and trajectory conditions to achieve precise control over video content dynamics. Tora is designed to fully leverage the scalability of DiT, allowing for the generation of high-quality video content across different durations, aspect ratios, and resolutions. The model excels in motion fidelity and physical world motion simulation, offering new possibilities for video content creation.

PIXART-Σ is a diffusion transformer model that directly generates 4K resolution images. Compared to its predecessor PixArt-α, it offers higher image fidelity and better alignment with text prompts. The key features of PIXART-Σ include an efficient training process, where it evolves from a 'weaker' baseline model to a 'stronger' model by leveraging higher-quality data in a process called 'weak-to-strong training'. Improvements in PIXART-Σ include the use of higher-quality training data and efficient label compression.